NLU

Natural language understanding comprises a wide range of diverse tasks such

有监督任务

• textual entailment, natural language inference,
• question answering
• semantic similarity
• document classification.

无监督任务

• Improving Language Understanding by Generative Pre-Training。先无监督训练语言模型，再对每个task做fine tune。